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ABSTRACT 

this report assesses the effectiveness of the Napa 
County (California) Instructional Skills Staff Development Program, 
focusing oti its impact on student achievement and student engaged 
rate in classrooms . The program^ providing training. for teachers and 
principals in administrative and ihstructibhal skills^ is examined 
for outcomes in two schools, both w^th lower socioeconomic students. 
Data were collected on students' time on task, teach^ 
instructional interactions , and classroom management , and on general 
organizational climate and demography. Analyses focus on four major 
areas: (1) engaged-time rates in classrooms, (2)_student achievement, 

(3) teacher implementation of the Instructional Skills program, and 

(4) correlations among the above three areas. Nine of the 14 teachers 
studied improved or maintained their ihstructibhal skills after 
returning to the classroom from the Ins^ 

Students in all but one classroom were more attentive posttest 
than during pretest visits. Further, achievement scores show that the 
lowest and highest performing students made progress after 
impiemehtatioh of the staff development program. For math, but not 
for read ihg, the program improved student engaged time and 
achievement. (JW) 



*********************************************** 

* Reproductions supplied by EDRS are the best that can be made * 

* from the original document. * 

*********************************************************************** 



EKLC 



_ U^. DEPARTMENT OF EDUCAnON 

.NATIONAL INSTITUTE OF EpUCATlON 

EOUCATIONAL.RESOyRCES INFORMATION 
_ . CENTER (ERIC) . . 

SL^This document has been reproduced at 
^ received from the persdii or organizatidr) 

orloinatlng it. 
i j Minor changes have been made to improve 

reproduction quaiity- 

• Points of view or opinipns stated in this docu- 
ment do not necessarily represent official NIE 
pdsilidh or policy; 



"PERMISSION TO REPRODUCE THIS 
MATERIAL HAS BEEN GRANTED BY 




TO_TUE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC);" 



m EVAttJATION OF THE NAPA COUNTY OFFICE OF_EDUCATION'S 
FOttOW THROUGH STAFF DEVELOPMENT EFFORT 
TO INCREASE STUDENT LEARNING TIME AND ACHIEVEMENT 



Jane A, Stall ings 

Director, Peabody Center for Effective Teaching 
Vanderbilt Utiiversity 
Nashville i Tennessee 



Paper presented at the Annual Meeting of the American Educational 
Research Association, New Orleans, April 1984. Study funded by NIE 
Grant ^fR-81-0024. 



This report describes and assesses the effectiveness of the Napa 
Bounty Ihstructibnal Skills Staff Development Program as it relates to 
student engaged rate and student achievement. 

The project was designed to accomplish two goals: to meet the 
immediate need to improve student achievement, and to build site-level 
resources to continue the process of instructional improvement after 
funding is no longer available. The focus of the prdfject is on the 
factors related to the improvement of instructional processes^ 
Academic Learning Time (ALT) and student achievements 

The project design focuses bti the teachers, the principals, and 
the students. The Staff Development Program is based oh many of the 
ideas of Madeline Hunter at the University of eaiifornia in tos 
Angeles. The program includes training for teachers in effective 
classroom management and instructional skills. It also provides 
training for the school principal. This training includes the 
supervision of itistructibri, planning, decisioti making, problem 
solving, and allocation of resources. Follow-up activities to assure 
successful applicationn of skills taught during training are also 
encouraged. Principals have been encouraged to leartl to structure 
meetings which will facilitate the exchange of information across 
grade levels regarding students and curriculum issues. 

The approach is grounded itl research and se&ms to be useful at 
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any grade level. It can be used with most curricalams and it does not 
demand any particular classroom physical arrangement in order to be 
Impiementedi in addition, the ihservice training strategies model the 
kitids of practices which teachers and principals are taught to use in 
classrooms at the school. This program encourages school personnel to 
work together toward a common goal: student achievement. 

The evaluation addresses the question of program effectiveness as 
illustrated by Figure 1. The staff development activities were 
designed to have ah impact oh the teachers' behavior in classrooms and 
on the principals' leadership skills. The improved teaching and 
leadership are expected to increase students' Academic Learning Timei 
The link between increased ALT and student achievement has been 
established (Fisher, 1978; Brophy, 1979; Stallings, 1978). Thus, to 
properly evaluate the effects of such a program, each element in 
Figure 1 must be assessed. 



Staff 
Development — 



elinical Supervision 



Teaching B e h a v ior^ 



Academic 
Learning Time 



Student 
Achievemeht 



Figure 1. The Evaluation Design 
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The evaluation was designed to be both summative and formative. 
As soon as possible, analysis of the data were shared with the project 
director, teachers, and principals. Surihg the past year^ student 
level engaged rate data were given to the teachers i This occurred in 
fall 1982 and spring 1983. The teachers fbuhd the information so 
useful that they requested a longer period of time to examine the data 
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arid discuisiB children who had high off task rates; 
SAMPLE DESeRiPTiGNS 

Both schools serve a low socio-economic status population and 
have been receiving Title 1 funds for several years; Both schools 
were required to give up these funds to accept the Follow Through 
project; While there are many similarities among the schools, there 
are also differences. 
School 

A new principal was hired at School #1 for the school year 
beginning fall 1982. The decision to change principals was made by 
the project director and District Super ihtehderit. The reasons for the 
change seemed to revolve around the former principal's difficulty in 
cbmrauhicating with the teachers and a lack of willingness to implement 
the project goals and objectives. 

The new principal in School #1 received his B.A. in History in 
1958 1 and his M.A. iti Educational Administration in 1963^ He taught 
fourth, fifth, and sixth grades for five years. He has served as a 
principal for 19 years in this district. 

He has been through the satne Instructional Skills and eiinical 
Supervision training that forms the program evaluated here, but at a 
different location and with different trainers. 

There were 26 aides working in the Bilingual, Special Education, 
Title i, and School Improvement Programs in 1981-82; A Special 
Education resource teacher worked 4-1/2 days a week. There were two 
Special Day Class Teachers for K-3: Language Disability and Special 
Education* 

Health personnel included: one nurse one da> a week; one Health 
Aide five days a week; one School Psychologist two days a week; and a 



Speech Therapist 4-1/2 days a week. 

The school had brie arid brie-quarter secretaries^ atid two 
custodians. 

Baseline data, spring 1982, were collected from li of the 16 
teachers at Schbol #1- Twb teachers taught kindergarten, three taught 
first grade, two taught second grade, two taught third (in combiriatibri 
with other grades), and two taught fourth grade. Due to the 
difficulty iri bbtairiing pretest data frbm kindergarten, a decision was 
made nationally to drop kindergarten from the sample. Thus brily nine 
teachers are included in these analyses; The average number of years 
these teachers had taught was 17^ with a range from six to 26 years. 
The average number of years the teachers had been teachirig in this 
schbol was eight, with a range from one to 18 years; 

All teacheris had completed between 45 arid 100 uriits of pdSt-B;A; 
graduate work; The teachers in this schbbl had rib inservice trairiirig 
for the past three years because of their participation in a national 
study bf drug educatibri. Two of the teachers had attended the project 
RAiSD training, which is ari inservice prbgram similar tb the brie beirig 
evaluated here. 

School #2 

The principal of School #2 has been involved in this project 
sirice its cbriceptibri iri the suimiler of 1981. He has supported his 
school's participation in the project arid has been wbrkirig clbsely 
with the project director to encourage support from the cbmmuriity, the 
schbbl bbardi and his teachers sirice the prbgram received approval in 
the fail of 1981. He has been to Washingtbn D.C. several times tb 
attend project meetings; 



This principal received his BiAi in History in 1973 and his M.A. 
in Educational Administration iti 1978. He served as a teacher in 
grades K-6 from 1973 to 1977* He was an ass:.starit principal of a 
year-round elementary school for two years and became principal of 
School #2 in 1979. He has been a principal at this school for four 
years. 

During the two years prior to this project, this principal went 
through the same inservice training in Instructional Skills and 
Clinical Supervision that is being evaluated. In addition, he was 
involved as an instructional skills trainer with the project director 
in the school years 1981-82 and 1982-83. 

There were five Title I aides * one Bilingual aide, a Special 
Education aide, one Special Day Glass teacher, one Title I Resource 
teacher, one Miller Unruh Reading teacher, one part-time Speech 
Therapist, one Learning Disability teacher (1-1/2 days per week) > and 
one Title Vil teacher (1 day per week) working at School #2 in 
1981-82. 

Data were collected for five of seveti teachers at School fft^ 
Baseline data, spring 1982, were collected from one kindergarten 
teacher, one first grade teacher, two teachers of second grade 
(combined with other grades) ^ and brie fourth grade teacher. However, 
the sample for the 1982-83 year is the following: one first grade 
teacher, one first /second, one second/third, one third grade, and one 
fourth grade teacher. The reported years of teaching experience 
ranged from five to 33 with an average of 16 years. The average 
riumber of years teaching at School #2 was nine, with a range from one 
to 18 years. 

Ail teachers had completed between 45 arid 78 iiriits of pbst-B.A. 
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graduate worki As a result of participation in a multi-county staff 
deveioptnetit program, all but one teacher had attended the RAISD 
workshops and other iriservice training in recent years* 
METHeDeteGY 

The contract called for selection or development of instruments 
to assess program outcomes at three levels: students^ teachers, and 
principals. The instruments were developed during Phase I of the 
study and modified in Phase II* The instruments developed for use in 
Phase II of the study included: the Time On Task (TOT) observation, 
instructional Skills (IS) observation^ Classroom Management Ratings, 
The Organizational Climate Description Questionnaire, and Teacher 
Attitudes and Demographics Questionnaire* All of the instruments 
remained the same from pretest to posttest except the tCt. 
Time O n^^^sk^ (TQX)-Qbservat^ton 

The object of the Time Oh Task observation instrument is to 
record a sample of all students' attending behavior or non-productive 
use of time during two periods of reading and two periods of math* 
This method provides ah engaged rate score for each child in the 
study. The engaged rate can be linked to student achievement data for 
reading and math. The following behaviors are considered Off-Taski 
Off-Task Behaviors 
e « ehatting: tow talking or whispering^ passing notes 

between students which pulls them off task 
D = Disruptive: Bothering a number of students, e»gi , loud 

talking, throwing things ^ pushing or fighting 
P » Personal Needs: Sharpening pencils, going to the toilet, 

getting a dritiRi getting papers or books 
U » Uhinvolved: Staring or sleeping 
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W « Waiting: 



Waiting with hand up for teacher's attention^ 



waiting for materials to be passed. 



Activitie s^ 

It is also of interest to know what was the expected activity 
when students were off taskt Were they supposed to be doing sesat work 
(silent reading or written work); listening to the teacher making 
assignments or organizing (getting papers and books but); listening to 
the teacher's instructions or explanation; reading aloud; taking part 
in a questions/answer period (children writing math problems on the 
board is included); or waiting in line or for materials. 
Grouping 

A third interest is to know whether the off task behavior 
occurred at a time when the student was expected to be attending 
within a small group (2-lD), a large group (U-tbtal), or working 
alone /independent work; 

Coding Procedures for the TOT 

A sweep is made for off task students each five minutes. As soon 
as the instructional period officially begins, the observer writes the 
time on the first sweep cbiutnn- Observers mark each student only once 
during a five minute sweep. They must watch the time carefully and 
start a new sweep at the next five minute interval. 

Achievement Testis 

Achievement tests were administered to the students by each 
teacher in May of each year. School ^1 uses the Stanford Achievement 
Test and School #2 uses the California Achievement Test. Schbbl 
and School #2 had made attempts to align their math curriculums with 
the concepts examined iti their achievement tests; 
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l as tract lonai Skills (IS) Observation 

To assess the level of teachers' use of the prescribed teaching 
skills, a classroom observation instrument was designed and field 
tested twice before collecting data; Refinements and review by the 
designer of the program^Madelihe Hunter ^ occurred after each field 
test. 

This instrument was designed to examine a teacher's ability to 
use certain specific ihstructibhal skills effectively when it is 
appropriate that they be used. The instrument exaSines the teacher's 
decision-making skills during the presentation of the lesson. The 
observer records how the teacher conducts four critical elements of 
the program: a set, instruction, guided practice and independent 
practices 

The critical issue for the observer is tlbt whether a certain 
element occurs , but : 

* Was there a need for the use of that element? 

* If there was a need , was it acted iipon? 

* If ah element was used, was it used effectively? 

The procedure was for observers to make anecdotal records of the 
lesson and then score the four elements on a rating scale. 
Observer Training 

Three observers were trained in a three-day session to collect 
data on the TOT in fall 1982. This training included three checks for 
ihterrater agreement in classrooms. The percent of agreement was 
>90%. Amazingly, the same three observers were available to collect 
data in the spring of 1983. A one-day training session preceeded 
spring data cbllectibh. These bbservers also rated the teachers oh 
Management SkiiiSi At School 91, each teacher was observed by two 
O 8 
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observers. At School #2 one observer collected the data, but 
liiterrater agreement with the evaluation project leader was computed 
on two occasions. Each time this agreement was found to be betwiBiah 85 
and 93 percent on all ratings. 

The Ihstructibhai Skills data were collected by six observers who 
were themselves trainers iti the Madeline Hunter Model. Thus^ they 
started their observer training with a thorough knowledge of the 
five-step lessbti plan which was being evaluated* The training 
occurred over three days. Training activities included discussions of 
operational definitions and checks of interrater agreement during 
three in-class bbservatibhs. Initially, agreement was hard to reach 
since each observer had a preconceived notion of what was good 
performance on each instructional skill. After much discussion arid 
delineating: bperatibnal definitions, acceptable agreement levels were 
reached (>.75 iriterrater reliability). All teachers were observed by 
two or more observers in the winters In the springy after a one-day 
retraining session, the same bbservers cbllected the posttest data^ 

Data Co lle ct i on Sched ule 

The schedule of data cbllectidn is provided in Table li 
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Data eoilection 1982 - 1983 



Spring 82 Fall 82 Winter 82 Spring 83 



Time Off Task 

Observations 

Instructional Skills 
Observations 

Management 
Ratings 



X 



X 



X 



X 



X 



Teacher 

Questionnaires 
Achievement 

Tests 



X 



X 



X 



X 



ANALYSIS AND RESULTS 

The analyses were conducted at three levels: child, teacher, and 
school* The analyses focused upon four major areas: (1) engaged rate^ 
(2) achievement, (3) teachers' implementation of the Instructional 
Skills program, (4) correlations between engaged rate, achievement, 
and program implementation. 

Engaged Rate 

Engaged rate was computed by observing each child once every five 
minutes for two reading periods and two math periods. If the class 
was fifty minutes long, there would be ten observations per period x 
four periods. Frequencies of off task behavior were recorded during 
the class period for each child i Off task (Q'i) frequency was then 
subtracted from the total number of observations to compute ati engaged 
(EN) irate i e.g.: 



An engaged rate was then cbtttpiited for each teacher by computing a mean 



Reading Total 20 observations - 4 0T ■ 16 EN 
Percent Engaged » 16 / 20 - 80 % 
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and standard deviation using the engaged rates of all the students in 
the class. In the fall, only total engaged rate scores were computed; 
iti the spring, scores were cdtnputed separately for reading atid ttlath. 
These separate math and reading engaged rates were then used in the 
correlations for reading achievement, math achievement, reading 
instructional skills, atid math ihstructidnal skills. 

Table 2 presents the mean percent of time students were engaged 
in each classroom during reading and math combined for fail and during 
reading atid math separated for spritig. The grade levels are iti 
ascending order for each school. 

Overall, the children in School #1 improved their engaged rate 
from fall 1982 to spritig 1983 by approximately 8 percent. When 
compared to fall scores^ all of the teachers' spring classroom engaged 
scores were higher in either reading or math except teacher Hi 
Dramatic improvemetit of engaged rates were found for the classrooms of 
Teachers and 1. No pattern was found in School #1 for 

students being more engaged at higher grades* The lowest engaged 
rates reported are for the kindergarten class and one third grade and 
one fourth grade class. All of these made significant improvements 
during the year, but remained lower in engaged rate than the other 
classes. Teacher H, whose scores were less good in the spring than in 
the fall^ seemed to have given up trying in the spring. This teacher 
felt the hail noises were very distracting for the children in his 
group* The school was designed for open space classrooms, but recent 
attempts have been made to provide individual rooms by arratigitig 
chalkboards and bookcases as dividers. These help with visual 
distractions, but the noise factor remains a probiem, especiaiiy whiie 
classes of child reti pass through the hallways. 
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Table 2 

Engaged Rates Fall and Spring 



Fall 198 2 



Spring 1981 
Reading 



Math 



Teacher 


Grade Level 




X 


SD 




X 


SD 


X 


oU 


semoh #i 




















A 


K 


29 


57% 


18% 


28 


70% 


19% 


72% 


15% 


B 


Ist 


27 


73 


13 


3b 


82 


13 


78 


15 


e 


1st 


30 


69 


14 


28 


87 


13 


82 


10 




2nd 


27 


81 


13 


27 


81 


11 


91 


8 


E 


2nd 


24 


73 


14 


27 


80 


14 


72 


16 


F 


3rd 


26 


74 


13 


27 


81 


13 


78 


16 


6 


3rd 


27 


59 


16 


27 


63 


24 


77 


17 


H 


4th 


29 


82 


10 


27 


78 


13 


76 


17 


i 


4th 


20 


64 


12 


21 


76 


10 


75 


15 


School Score 




294 


70 


16 


238 


78 


16 


77 


15 


SCHOOL #2 




















A 


Ist 


25 


82% 


8% 


27 


78% 


12% 


83% 


14% 


B 


1-2 


27 


77 


10 


21 


82 


10 


80 


8 


e 


2-3 


23 


86 


10 


22 


81 


16 


89 


11 


i) 


3rd 


19 


91 


8 


23 


91 


9 


89 


8 


E 


4th 


26 


87 


10 


15 


87 


12 


84 


14 


School Score 




152 


84 


10 


134 


83 


1 3 




42 



12 



14 



School #2 started in the fall with a significantly higher engaged 
rate (84%) than did School #1 (70%), Teacher D had consistently high 
scores (91%, 91%, 89%). The teachers spring scores only fluctuated 
three to five percentage points from the fall scores i The overall 
school score did hot change. There was a trend in this school for 
teachers of younger children to have lower engaged rate than teachers 
of older children. 



Achievement 

This longitudinal study has the rare opportunity to monitor 
students' achievement over several years. Successful schools, as Ron 
Edmonds points out (19815, are those which help all children make 
progress, low achieving and high achieving* In this study we have 
recorded the percent of children performing in each quartiie of 
achievement test results. Over time^ if the instructional Skills 
program is implemented and if the program is effective, there should 
be fewer students in the lower quartiles and more in the upper 
quartlles. 

The achievement test data gathered for Schools #1 and #2 have 
been examined for such trends (see Table 3). Typically, the children 
in the two schools of this study have bti the average scored well below 
national norms* In spring 1983, School #1 had 5% fewer children iii 
the lowest quartiie in mathematics, and 4% more in the highest 
quartiie than in spring 1982. A similar trend is found in reading: 
4% fewer in the lowest quartiie and 4% more in the highest quartiie 
when spring 1983 scores were compared to 1982 scores. In real terms ^ 
it means that approximately ten children have shifted their scores 



Table 3 

Percent of Students in Each Achievement Quartile 
Spring 1982 and 1983 

1st Quartile 2nd^ Qu a rtii & 3rd Qu artile 4th Quartile 



O^ to 25% 
1982 1983 

ICHOOt #1 Total Tested 
Reading 
Math 

CHOdt H 
Reading 
Math 



26% to 50% 51% to 75% 76% to leOZ 
1982 1983 1982 1983 1982 1983 



234 
232 



39% 
30% 



34% 
26% 



25% 
27% 



27% 

28% 



19% 

24% 



18% 
23% 



m 
in 



21% 
23% 



131 
130 



21Z 
16% 



25% 
11% 



27% 
22% 



21% 
17% 



27% 
34% 



25% 
36% 



25% 
28% 



25% 
36% 
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upward in reading and ihath from the lowest performing group arid teri 
others have shifted their scores into the highest pesrformirig group. 

The big shift for School #2 came in tnathematics. In 1982 there 
were 38% of the children perfbrmirig below the 50th perceritile arid iri 
1983 only 28% were performing below the 50th percentile. There were 
8% more children performing in the highest quartile (>75%), The 
progress in math might be attributed to the staff's efforts to become 
more aware of what math concepts were tested on the Gaiifbrnia 
Achievement Test. One component of the teacher training program 
included math curriculum aligrimerit activities. These types of 
activities were not included for reading, and reading scores at School 
#2 remained at approximately the same level for 1982 and 1983^ 

Differences in 1982-1983 Achievemerit Test Scdres-fey-Cl a s s rooms 

Reading and math scores for each teacher were examined for 
differences from Spring 1982 to Spring 1983, using a two-tailed T-test 
for significance. The attrition rate is quite high iri these schools 
and thus there were approximately two-thirds of the students with a 
complete set of test scores. Of the 14 classrooms of children 
examined i only those childreri with teachers B, C, and F of School #1 
made significant gain in reading during that ischbol year (see Table 
4). Children in classrooms of teachers B and D in School #1 made 
sigriificarit gairis iri mathematics* 

efiildren in the classrbbm bf teacher I iri School #1 had 
significantly lower scores on the post test. This teacher was shifted 
from ari admiriistrative role to the classroom for the last two years 
befbre retirement. This teacher ^ as cari be seeri bri the itnplemeritatibn 
section, improved on instructional skills^ but the trairiirig came tbb 
Q iate iri the schbbi year to counter the effect of this teacher's 
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TABLE n 



TESTS FOR DIFFERENeE IN Kmm AND MATH SCORES 
PRETEST SPRING 1982 POSTTEST SPRING 1963** 



SCHOOL n 



A 


im 

18 


PRETEST 
m.2 35.3 


POSTTEST 
X SD 
35.6 5T.i| 


PRETEST 
15.6 lu.1 


pesHEsr 

19.8 Z5.1 


B 


22 


31.9 


25.2 


MS .9* 


29.8 


37.9 


25.9 


18.2* 


26.6 


e 


m 


38.5 


3i|.9 


52. !• 


35.8 


18.1 


35.6 


52.8 


36.6 


D 


22 


m.9 


23.6 


MO.S 


18.8 


50.9 


21.8 


61. 6* 


19.9 


E 


21 


53.6 


33.0 


58.9 


30.1 


56.8 


31.1 


62.0 


32.6 


F 


23 


MQ.9 


25.7 


M7.2* 


21.9 


15.7 


27.1 


52.9 


27.6 


6 


23 


2M.2 


2n.b 


25.1 


26.8 


30.6 


21.7 


26.9 


23.1 


H 


26 


ko.s 


27.0 


H3.3 


21.7 


53.9 


27.1 


52.5 


25.1 


I 


15 




25.7 


35. 5» 


25.2 


55.5 


27.8 


11.0* 


29.3 



ScHObt #2 

TEAGHER 



A 


10 


51.2 


26.9 


3i.r 


23.9 


15.9 


16.1 


11.5 


28.1 


B 


12 


59.3 


29.1 


51.9 


29.8 


62.1 


20.9 


68.5 


19.3 


e 


11 


67.9 


25.2 


52.0 


30.5 


68.2 


31.5 


67.5 


17.1 


D 


11 


10.7 


29.1 


11.6 


25.0 


67.1 


25.1 


68.1 


22.9 


E 


7 


58.3 


21.6 


52.6 


23.9 


71.0 


13.2 


68.6 


16.7 



• p.<.b5 
2 Tailed pRbiAiiLiTY Test 



negative impact upon the students^ 

The children in the classroom of teacher A in School ^2 had a 
significantly iov^er score in reading oh the pbsttest. This may be 
explained in part by the fact that although 25 children had pretest 
scores, only 10 of these children remained for the spring 1983 
testing^ In this school district it appears that the upwardly mobile 
families tend to move on, leaving perhaps those children who score 
lower oh tests. 
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^udetit Achlevetnent Comp ared^witti^ngage<i Rate 
Are the students who achieve at the higher levels the same 
students who are on task more of the time? Are children who have 
limited English speaking skills off task more of the time? To examine 
these and other relationships » children's engaged rates were organized 
by deciles and their achievement test scores by quartiies in a two by 
two table (see Table 5). These data are presented for ihath and 
reading for each school. (Similar data are available for each 
teacher's classroom from the evaluator.) 

Of the 50 students perfdrffling in the highest reading iquartile 
(75%-100%) in School #1, 24 were off task less than 10% of the time. 
Surprisingly, twelve of the high achieving students were off task 20% 
or more of the time* Iti fact, one of these students was off task 50% 
or more of the time. If this child is achieving at this level without 
using the in-*ciass time, it may be that the lessons are too easy* A 
similar pattern Ls found for two high achieving math students. 

Approximately bhe-^fburth of the lowest achieving reading and math 
students were engaged in their work 90% or more of the time. This 
indicates that they were listening to the teacher and working on their 
lessons. However ^ according to the results of the achievement test| 
their efforts were in vain. It is important that teachers be aware of 
these children's effort to stay on task and provide appropriate 
lessons so that they can make progress. Perhaps by the next testing, 
some of these high attending low achieving children will move upward 
to the 2nd quartile. 
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Table 5 

Reading and Math Achievement Qaartiies 
and Time Off Task Deciles by School 









Reading 


Off Task 


Deciles 








1 Aft. on 
luU— 70 


Qd— fin 


79-70 


69-60 


59-50 


< 5U 




vjUamie 


u— lU^ 


1 1 —ony 
1 1— ZU^ 


21-30% 


31-40% 


41-50% 


3U— lUU^ 


Total 






OA 


16 


14 


7 


4 


OA 

94 






1 £ 


13 


5 


4 


4 


DO 






9 


13 


2 


5 


0 


43 




fix 


14 


4 


4 


3 


1 


50 


rotax 


o!> 




4b 


25 


19 


ri 

y 


243 








Math Off Task Deciles 






(juartiie 


















ID 


14 


17 


10 


1 


o 
J 


01 


*ii ~C A*S 


2i 


l3 


14 


11 


6 


2 


d7 


c i ^ c 


24 


12 


9 


7 


i 


2 


55 


7o-100% 




22 


10 


4 


i 


2 


54 


Total 


lb 


61 


50 


32 


9 


9 


237 


seHeot #2 






















Reading Off Task 


Deciles 






Quartile 


0-10% 


1 1-20% 


21-30% 


31-40% 


41-50% 


50-100% 


Total 


0-25% 


9 


8 


6 


0 


1 


1 


25 


25-50% 


9 


7 


5 


1 


0 


0 


22 


51-75% 


9 


8 


7 


0 


1 


0 


25 


76-100% 


14 


$ 


1 


i 


i 


6 


26 


Total 


4i 


32 


19 


2 


3 


i 


98 








Hath Off Task Deciles 






Quartile 
















0-25% 


8 


2 


2 


0 


1 


0 


13 


26-50% 


4 


7 


6 


2 


6 


i 


20 


51-75% 


i6 


11 


9 


4 


d 


e 


4d 


76-100% 


' 14 


15 


10 


^ 1 


i 


d 


4i 


Total 


42 


35 


27 


7 


2 


i 
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The achievement and engaged rate patterns for children in School 
#2 are more predictable i Over one-half of the 26 children who were 
achieving in the top quartile of reading were bti task 90% or more of 
the time and only three of the high achieving students were off task 
more than 20% of the tiraei The only student off task more than 50% of 
the time was a low achieving student. Of the 41 children in School ^2 
who were engaged in their reading 90% or more of the time, 14 were in 
the highest achievement quartile and the remaining 27 were evenly 
distributed across the Ist^ 2hd , and 3rd quartiles (see Table 5)« 

Overall, the children in School ^2 were better prepared in math# 
Over 70% of the 114 tested were above the SOth percentiles Of these 
81^ 30 children performing above the national hbrifi were engaged in 
their work 90% or more of the time. Interestingly, eight of the 13 
children in the lowest achievement group (Ist quartile) were also on 
task 90% or more of the time during the observed inath classes. 

Apparently there is not a perfect correlation between high 
engaged rate and high achievement or off task behavior and low 
achievement. Some children pay attention and still achieve poorly, 
others are off task and still do well academically. In either case 
there is something for teachers to learn about appropriate lessons and 
motivating children to use their time productively; 

A .31 correlation (p<.001) was found between the math and reading 
engaged rate scores for the 340 children observed in both subjects. 
Eveti though significant, a •SI correlation is low and tiieans that a 
considerable number of children were hot engaged at the same rate in 
both subjects. Figure 2 illustrates this correlation* 
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Figure 2 



A« 1 

B = 2 students 
o" ~ "i students etc. 



The letters on the figure represent the number of children at any 
one poiiitj e.g.^ A*l child ^ G^7 children. In the upper left corner of 
the figure are those children who were highly engaged in reading but 
not in math. In the lower right corner are those children who were 
highly engaged in math but riot iri readirig* The erigaged rate of these 
children must be related to the subject rather thari to lack of ability 
to pay attention during class time. Teachers' awareness of these 
students could perhaps help them iricrease erigaged rates iri the other 
subject by prbvidirig more appropriate leissoris. Children iri the lower 
left corner are not engaged in either reading or math. These children 
most likely rieed help iri building cbriceritratibri skills to help them 
stay engaged during academic class time. 

Iri order to benefit most from the time spent on academic tasks, 
kiridergarteri through fourth grade children rieed to be erigaged at least 
86% of the time, in this study, 137 of the 351 children observed in 
math did not meet this criterion. Of the 357 children observed in 
readirigi 127 were erigaged less thari 80%. This suggests that teachers 
need to consider their organizational strategies and plari lessons that 
will help the chronically off task children use the available time 
more efficieritly. 



iiimited English Speaking Ghiiareh 

The limited English speaking children at School #2 were taught in 
special classes. Thus^ they were hot observed in the reading or math 
classes and were not included in the achievement test sample. 

Fifty-eight LES children were included in School #l's sample. Of 
these ^ 75% tested below the 25th percentile on the reading test^ 
compared to 28% of their English speaking counterparts who tested at 
that level (see table 6). this poor reading performance on the part 
of the LES children is hot surprisihg giveh their problem in 
understanding the language, they tested somewhat better in math, 
which depends more bti symbols and less dti language (see Table 6). In 
spite of their difficulty with the lahguage^ the LES childreh were as 
attentive during the reading classes as were the other childreh. 
Approximately one-third of all the children were engaged 90% or more 
of the time. Ih math^ 40% of the LES childreh were ehgaged over 90% 
of the time compared to 36% of the other children who were engaged at 
that rate. 
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Table 6 

LIMITED ENGLISH SPEAKING STUDENTS 
:eiiding-^nd Math Achlevetnent and Tittle Off Task 



idlng Achievement Quartiles 






26-50'/ 






Oi-iflOX 


Total 


Limited English Speaking 






12 


3 




6 


59 


Non blmlted English Speaking 


-M 




J! 


_43 




_52 


m 


Total 


103 




71 


46 




52 


272 


h Achievement Quartiles 
















Limited English Speaking 


31 




18 


6 




3 


58 


Non Limited English Speaking 


_52 




Jz 


51 




54 


214 


Total 


83 




75 


57 




57 


272 


ding Time Of! Task Deciles 




ii-m 2i-m 


41-40^ 


51-50^ 


5Q-mt 


total 


Limited English Speaking 


i7 


13 


9 


6 


5 


2 


52 


Non Limited English Speaking 


-6a 












-192 


Total 


85 


59 


46 


25 


26 


9 


244 


h Time Off task Deciles 
















Limited English Speaking 


21 


11 


13 


4 


2 


1 


52 


Non Limited English Speaking 


55 


50 


37 


28 


8 


8 


m 


Total 


n 


61 


50 


32 


10 


9 


238 



28 

7 
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Instructional Skil ls 

The Instructional Skills program is comprised of four separate 
skills: Anticipatory Set^ Ihstructibh^ Guided Practice ^ and 
Independent Practice. Each teacher was observed and received a score 
in reading and math for each component and a total score which was the 
sum of all coTdpohehts. The total number of points that could be 
received was 88 i Before looking at individual teachers' scores, it is 
of interest to see which components were best implemented before and 
after the training period • 

To compute the percent of implemehtatioh for each component, the 
combined teacher scores were divided by the homber of teachers x the 
number of possible pditits. Table 7 displays these winter atid spring 
implemehtatioh scores. 

Table 7 



Winter and 


Spring I 


mplemeiitatibn 


Scores 




Minter 


Spring 




Math 


Reading 


Math Reading 


Anticipatory Set 


41% 


50% 


63Z B3% 


instruction 


57% 


59% 


lit 67% 


Guided Practice 


56% 


52% 


71% 62% 


Ihdepehdeht Practice 


44% 


45% 


63% 61% 



Except for "Ihdepehdeht Practice" and "Set" ih math^ the teachers 
were implementing approximately 56% of the program variables before 
the training program started. The training was provided during 
February^ March^ ahd April. The firist cbmpbheht taught was 
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"Aiiticipatbry Set" and this component was implemented 21% better in 
math and 13% better in reading after the training. "Iristriictibh" was 
taught next in the training program, and this improved by 
approximately 10% iti both reading and math. 

"Guided Practice" was taught late in April just before the post 
observations* The highest implementation score on the table is in 
tnath "Guided Practice". Guided practice requires teachers to check 
children for understanding and monitor their work to be certain they 
are doing it correctly. The structure of math lessons seems to lend 
itself to guided practice, e.g., it may take the form of large or 
small group drill and practice or the teacher and students dbitig 
assignments on the chalkboard. Reading lessons are more likely to 
have the teacher instructing one small group while other students are 
doing independent seat work. The guided practice cbmpbherit is more 
likely to be omitted in reading class. 

"Independent Practice" was taught late in the spring training 
program. Nevertheless^ the teachers imprbved 19% in math and 16% in 
reading on this components It means that children were given 
independent work after the teachers had checked for understanding and 
were certain they could db the wbrk albhe and hbt make errors. 

For only three months of training, the implemehtatibh change 
scbres are impressive, but allow for continued improvement in Phase 
III. Of course i within each school there were teachers who 
implemented the program at very high levels and those who did hot. 
The ranges were from 86% by Teacher B in School #2 to a low of 29% by 
Teacher H in Schbbl #1. The teachers' pretest and posttest raw scores 
are presented on Tables 8 and 9. 
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TABLE 8 



rNSTRUeTIQNftt SKILLS IMPLEMENTATION W INTER AND SPRING SeORES - SGflOOL »1 



READING 
Teacher 



School 
Average 

rin I n 

Teacher 





ANTIEIPATORY 




















SEI 




INSTRUCTION 


GUIDED 


PRACTICE 


INDEPENDENT 








WTR 82 


SPR 85 


WTR 82 


SPR 83 


Orb oo 

WTR 82 


CDb OT 

SPR 83 


PRACT4€E 
WTRTT^R 83 


TOTAL 
WTR 82 SPR 83 


A 


7.5 


15.0 


25.5 


28.0 


22.5 


26.0 


8.0 


8.0 


63.5 


77.0 


B 


9.5 


11.0 


20.5 


o 1 n 
21.0 


15.5 


8.0 


1.0 


8.5 


19.5 


18.5 


C 


1.0 


13.0 


1 o n 
19.0 


27.0 


18.0 


23.5 


g.d 


5.0 


50.0 


68.5 


D 


8.5 


12.5 


zi.O 


30.5 


18.0 


26.0 


7.5 


12.0 


55.0 


81.0 


E 


^.5 


8.5 


17.5 


14.5 


16.0 


6.5 


1.0 


3.5 


39.0 


33.0 


F 


0.5 


1.0 


3.5 


ICC 
15.5 


0.0 


13.0 


0.0 


2.0 


1.0 


31.5 


G 


6.5 


9.5 


21.0 


19.0 


19.0 


17.5 


3.5 


2.5 


50.0 


18.5 


H 


8.0 


9.0 


21.0 


12.5 


8.0 


1.0 


5.0 


6.0 


12.0 


31.50 


I 


h h 


8 0 


1.0 


11.0 


2.5 


2.0 


1.0 


6.5 


7.5 


30.5 




5.0 


10.0 


18.0 


22.0 




m .u 


5.0 


6.0 


11.0 


52.0 


A 


7.5 


12.5 


17.0 


21.0 


15.0 


12.5 


6.0 


7.0 


15.5 


56.0 


B 


2.0 


13.0 


10.5 


23.0 


13.0 


25.0 


0.0 


7.0 


23.5 


68.0 


C 


4.0 


13.0 


18.5 


27.5 


16.0 


25.0 


1.0 


12.0 


39.5 


77.5 


D 


11.0 


13.0 


18.0 


30.5 


20.0 


25.5 


1.5 


7.5 


53.5 


75.5 


E 


3.5 


7.5 


23.5 


22.5 


22.0 


22.0 


3.5 


6.5 


52.5 


58.5 


F 


3.0 


8.0 


11.5 


19.5 


3.5 


21.0 


3.5 


7.5 


21.5 


56.0 


G 


3.0 


3.5 


11.5 


16.0 


8.0 


11.5 


2.0 


3.0 


21.5 


37.0 


H 


8.0 


3.5 


19.0 


6.0 


16.0 


0.0 


8.0 


2.5 


51.0 


12.0 


I 


4.0 


7.0 


17.5 


19. 


15.0 


8.0 


0.0 


0.5 


31.5 


35.0 




5.9 


g.Q 


19.0 


21.9 


11.0 


17.0 


2.0 


5.0 


10.0 


55.0 
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INSTRUCTIONAL SKILLS IMPLEMENTATION WINTER AND SPRING SCORES - SCHOOL »2 

ANTICIPATORY 

SET INSTRUCTION 6UIDED PRACTICE 



INDEPENDENT 
PRACTICE 



TOTAL 



READING 




WTR 82 


SPR 83 


WTR 82 


SPR 83 


WTR 82 


SPR 83 


WTR 82 


SPR 83 


WTR 82 


SPR 92 


TEAGHER 


A 


13.5 


11.0 


19.0 


25.0 


17.0 


22.0 


5.5 


11.5 


55.0 


69.5 




B 


15.5 


9.5 


30.5 


28.0 


2M.5 


26.0 


11.5 


9.d 


82.0 


72.5 




C 


W.O 


12.5 


19.0 


18.0 


18.5 


18.5 


5.d 


8.d 


56.5 


57.0 




D 


10.5 


11.5 


241.0 


27.5 


22.0 


241.0 


8.d 


Id.d 


641.5 


73.0 




E 


11. d 


6.5 


22.5 


21.5 


19.5 


241. d 


5.d 


Id.d 


58.0 


62.0 


SCHOOt 

Average 




13.0 


10.0 


23.0 


241.0 


19.0 


22. d 


7.d 


Id.d 


62.0 


66.0 


MATH 
























TEAGHER 


A 


8.0 


11.5 


17.5 


28.0 


1M.5 


23.5 


8.5 


8.5 


418.5 


71.5 




B 


M.O 


13.0 


26.0 


26.5 


25.0 


26/5 


Id. 5 


11. d 


75.5 


77.0 




C 


M.O 


13.5 


20.0 


0.0 


18.5 


241. d 


id.d 


9.d 


62.5 


416.5 




D 


w.d 


10.5 


25. d 


28.5 


19. d 


27. d 


9.d 


id.o 


67. d 


76.0 




E 


8.0 


11. d 


18.0 


26.5 


15.0 


241.5 


8.d 


141.0 


419.0 


76.0 


Sghool 
Average 




11.0 


11.0 


20.0 


21.0 


18.0 


25. d 


9.d 


10.0 


58.0 


67.0 
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Overall I the teachers iti School #1 improved their implementatibn 
scores somewhat oh each of the four variables in reading and math 
classes. Teachers A, C, D, F, and I improved impressively in using 
"instructional Skills" iti reading. Teachers 6^ and E improved all 
four variables £h implementing the program in math. Teacher D 
implemented the program better in reading and math than did other 
teachers and also had the hir^ t student engaged rate of all teachers 
in the study (91%). 

Several teachers in School #i made very little improvement and 
some teachers (G and H) actually did less well iti the spring thati iti 
the winter. These teachers also had a higher percent of children off 
task than other teachers i it appears that teachers who implement the 
model well also were able to keep studetits dti task duritig the 
instruction periods. 

In School #2, the principal had been trained in Instractionai 
Skills duritig the previous years atid had shared some of the ideas with 
his teachers, consequently the teachers started in the winter with 
higher implementation scores than did teachers in School #1. However, 
by spring School #1 had caught up dti "Set" atid "Itistruction" in 
reading. School 1f2 continued to be higher on "Guided Practice" and 
"Independent Practice" in both reading and mathi 

Otie ationtaly iti School #2's data is Teacher B. She started the 
study with scores higher than any other teacher. She had received 
some training in the instractionai Skills program daring the previous 
years. Although her scores stayed quite high, they were not 
maintained at the high level recorded oh the wihter preobservatibh. 
Another anomaly occurred with Teacher 6 in math. She had 20 points 
for Instruction during the pretest and 0 points on the 
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postobservatlon; The observer's log indicated that her lessons in 
math for the two observed days were review: new information or 
concepts were not introduced, in reading her score for Instruction 
remained approximately the same iti winter and spring. 

Correlations 

To examine the relationship between student engaged rates ^ 
achievement test scores^ gain on achievement tests, and teachers' 
instructional skills, Spearman rank order correlations were computed 
using the 14 teachers for whom there were complete data sets. The 
cbrrelatibhs were computed separately for reading and math. The 
results of these analyses are presented in Tables 10 and 11; 

There were 12 significant relationships found between student 
engaged rate^ i983 math achievement test scores, and Instructional 
Skills* Total instructional Skills scores correlated (r»;73) with 
engaged rate, (r«.81) with 1983 achievement test scores, and (r«.65) 
with gain in math. These are impressive figures even with only 13 
teachers in the sample; 

As may be expected, the Instructional Skills total score 
correlated significantly with all four subscaleis and the subscales 
tended to correlate with each other; Guided Practice seemed to be one 
of the most effective components itl that it related impdrtahtly with 
1983 math scores ^ math gain scores, and engaged rate. Independent 
Practice related to 1983 math scores and engaged rate; instruction 
related only to engaged rate and Set related only the math gain 
scores. 
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Table 10 

READING Speatiiian CDrrelatlcti Coefflcletits, Spring 1983 



Mean Mean Instructional Anttcipatory Guided Independent 













Skills 


Set 

U^JL 


instructioji Practice 


Prarttce 


Engaged 


r = 


.65* 


.35 




.43 


.14 


.46 


.45 


.-43 




p < 




21 




.12 

lib 


.63 

I V J 


.09 


.11 


.13 


Mean 


r = 










- .13 


;14 


;28 




/itilig vcinciiL 


P i 




-A? 








;64 


;34 


-7i 


Heaii 


r = 








- 46 


..00 


- .07 


- .19 


- .44 




P i 








.58 


1 00 


.81 


.51 


.12 


Instructional 


r = 










.69* 


;97* 


i95* 






P C 










01 


;0i 


.01 


• VII 


Anticipatory 


r ' 












.64* 


.55* 


.33 




P c 












.oi 


.04 


.25 


Instruction 


r = 














;94* 


;64* 




P t 














.01 




Guided 


r = 
















CO* 


Practice 


















.03 












Standard 










Variable 




Nunib 


er 


Mean 


Deviation 


Median 


MifliffiUffl 


MaxinuuB 




Engaged Rate 




14 




80.13 


7.03 


80.77 


63.35 


91.30 




Mean Achievement 




14 




45.54 


9.60 


47.96 


27.85 


59.26 




Mean Gain 




14 




(TV / 1 

- 0.61 


i ts ^ c 

10.35 


- 1,61 


- 23.10 


17.96 




Instructional Skills 


14 




56.21 


18.30 


59.50 


30.50 


81.00 




Anticipatory Set 




14 




10.11 


2.86 


10.25 


4.00 


15.00 




Instruction 




14 




21.57 


6.11 


21.25 


12.50 


30.50 




Guided Practice 




14 




17.21 


8.82 


20.25 


2.00 


26.00 




Independent Practice 




14 




7.32 


3.20 


8.00 


2.00 


12.00 





^J'^^gnif leant at .05 or above 



Table 11 





MAfil 


1 - 


Spearman CorrelatiDn Coefficients, 


Spring 1983 




* 




Mean 




nean instructional 


Acnievem 


eht 


Guided 


independent 


Achievement 




Gain Skills 


.Set. 


_ listrnrttc 


m Practice 


Practiefr- 








• JO /1 3^ 


-iii 

• jI 


.52* 


.76* 


.72* 


Rate p < 


.01 




.20 .oi 


.06 


.05 


.01 


.01 










-11 




.77* 


-M* 


Achievement p / 






.03 ;6i 


.69 


.07 


.01 


M 


ricail I - 










.32 


.61* 


.43 


Gain p 






.02 


.02 


,27 


.03 


.12 


Tnstriiri'ional^ t = 








.01'^ 


;85* 


.90* 


;87* 


Skills p ^ 








.01 


M 


.01 


.01 


i\u ticipdcory r - 










.37 


.70* 


.62* 


Set p ^ 










.19 


.01 


.02 


instruction r = 












.80* 


.56* 














.01 


.04 


Guided r = 














.74* 


Practice p ^ 














.01 








Standard 










Variable 


I 




ler Mean Deviation 


Medlar 


1 Minlffiuffi 


Maxiffluii) 




Engaged Rate 




U 


80,^6 6.18 


79.18 


71.85 


90.53 




Mean Achievement 




14 


52.82 16.11 


54.10 


2i.lS 


78.19 




llcqil uaJlU 




U 


2.06 7.60 


2;67 


- 14.60 


13.48 




Instrtjctional Skills 




13 


59.77 20.54 


68. OD 


12.00 


77.50 




Anticipatory Set 




14 




11 1^ 
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Independent Practice 
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7.59 


0.50 
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^y'snif leant at .05 or above 
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Overall we must conclude that the use of the Instructional Skills 
model was effective in helping children stay engaged during math 
classes and in promoting student gains in mathematicis. 

For reading there were very few significant correlations. 
Reading gain was hot even related to 1983 achieveraerit test scores 
which cast some doubt upon the validity of the test scores. Engaged 
rate was related to spring 1983 achievement test scores (r».65) ljut 
ip/as hot related to reading gain scores. There were btily low 
correlations between engaged rate and three of the Instructional 
Skills variables: Ihstructidn (r»«43), Guided Practice (r«.45), and 
Independent Practice (r».435- Total Ihstructidhal Skills did 
correlate with all four subscaies, and the four subscales correlated 
with each other ^ but hot with student achievement in readings 

The explahatioh for the differehces ih the findings for reading 
and math may be in the different procedures used to teach math ahd 
reading. A reading lesson is likely to have several objectives, e.g., 
learhihg to incode and decode syllables and words. Multiple 
objectives may, in fact, be confusing to childreh. The Set ahd Guided 
Practice may be embedded iti the instruction. Teachers tend to have 
set routines for teaching several reading groups and these procedures 
may not have been very amenable to change. Apparently it was easier 
to Itlcbrpdrate the Instructional Skills into math lessons because, on 
the average i teachers improved reading implementation by 9.3 points 
from winter to spring, while oh math they improved by 24 pbihts during 
the same time period. 
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SUMMARY AND CONCLUSIONS 

The Napa County Follow Through version of Madeline Hunter's 
Instructional Skills model of education cati be observed objectively 
and implementation in the classroom can be measured • The bbservatibh 
system developed for the project was sensitive enough to measure 
teachers' use of fbiir model cbmpbhehts befbre atid after the training 
program. Althbugh the Hunter model bf educatibh has been used widely 
throughout the United States, it has not been previously 
systematically observed so that the quality atid quantity of the 
implemehtatibh were measured* This study bf implemehtatibh makes a 
major contribution toward validating the Hunter model. 

Nine bf the 14 teachers iti the study improved or maintained their 
use of instructional skills to ah acceptable level. 

Student engaged rate was measured in the fail and spring for 
every child itl the study; Students in all of the classrooms except 
one were more attentive to their lessons in the spring bf 1983 than in 
the fall of 1982. 

As in studies by Brbphy atid Evertsbti (1974), atid Stallitigs 
(1975), student engaged rate and achievement in math and reading were 
significantly correlated (p < .01). With such correlations, it is 
still impbrtarit tb remember this is tibt a perfect relatidtiship: 
thirty-two children testing below the 25th percentile were found tb be 
on task 90% or more of the timei Since none of these children were 
identified as beitig utiable tb learn, it appears that the lessons may 
have been inappropriate. Another five children were off task 50% br 
more of the time and were testing above the 75th percentile. Their 
lessons may also have beeti itiappropriately easy; Imagine what they 
Q nilght db if they gave gbbd attention. 
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Another fitiditig was that engaged irate is hot cbtistatit across 
subject matter. Some students highly engaged in reading classes were 
not so highly engaged during math period. Other students highly 
engaged in math were off task often during reading. These studehtis' 
propensity to stay on task may be conditioned by their interest in tlxe 
subject matter. Another interesting analysis of the data was 
conducted by Rbgbsa (1984). He looked at the duration of engaged rate 
for first grades and fourth grades, the first grades tended to be on 
task for ten minutes and then off task, the fourth grades duration 
rates were longer (20 mihiites). Findings such as these could have 
important implication for planning lessons and length of time 
allocated to one activity. 

Achievement gains for each school were examined by comparing the 
change in the number of students in each quartile of achievement in 
1982 and 1983. School #1 had 5% fewer students below the 25th 
percentile in 1983 and 4% more above the 75th percentile. Students in 
School 92 stayed about the same in reading but made shifts upward in 
math in the lowest quartile and the highest quartile. These data 
suggest that the lowest and highest performing students are making 
progress, particularly in mathematics* 

The major purpose of the study was to examine the relationship 
between student engaged rates and achievement in inath and reading and 
program implementations All of these relationships will be examined 
over the three years of the projects 

The Spearman correlations confirmed hypothesis regarding the 
positive relationships between student engaged rate and achievement in 
math and reading. Interestingly^ the hypothesis regarding positive 



relatibhship between engaged ratei student achievement dtid program 
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impiementation was confirmed for math bat not for readings 

At this point, it is hard to explain the difference in findings 
for reading and math. It may be that the more linear structure of 
math makes it easier to implement the components of instructional 
skills and make it easier to observe than is reading. Reading tends 
to be taught in several small groups and the four model cbmpdhehts may 
not be implemented in each group or it may be harder for the observer 
to fblldw and record the many changes of activities in a reading 
class* In the latter case^ the error is in the measurement. 

Fortunately, this analysis using the same instruments will be 
conducted for data collected iti 1983, 1984 and 1984-1985. Control 
classrooms will be included in these studies. If the findings of this 
study are replicated, we will have more confidence in the results, and 
conclude that Hunter's Instructional Skills model is effective for 
math and not for reading. At this time^ it is premature to make such 
a conclusion and our finding must be regarded as tentative. 
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Another finding was that engaged rate is not constant across 
subject matter. Some students highly engaged in reading classes were 
hot so highly engaged during math period. Other students highly 
engaged in math were off task often during reading. These students' 
propensity to stay on task may be cbtiditibhed by their interest in the 
subject matter. Another interesting analysis of the data was 
conducted by Rogosa (1984). He looked at the duration of engaged rate 
for first grades atid fourth grades. The first grades tended to be oh 
task for ten minutes and then off task. The fourth grades duration 
rates were longer (20 minutes). Findings such as these could have 
important implication for planning lessons and length of time 
allocated to one activity. 

Achievement gains for each school were examined by comparing the 
change iti the tiumber of students iti each quartile of achievement in 
1982 and 1983. School 91 had 5% fewer students below the 25th 
percentile in 1983 and 4% more above the 75th percentile. Students in 
School #2 stayed about the same in reading but made shifts upward in 
math in the lowest quartile and the highest quartile. these data 
suggest that the lowest and highest performing students are making 
progress^ particularly in mathematics. 

The major purpose of the study was to examine the relationship 
between student engaged rates and achievement in math and reading and 
program implemehtatibti. All of these relationships will be examined 
oyer the three years of the project. 

The Spearman correlations confirmed hypothesis regarding the 
positive relationships between student engaged rate atid achievement iti 
math and reading. Interestingly ^ the hypothesis regarding positive 
relationship between engaged rate, student achievement and program 




implementation was confirmed for math but hot for reading* 

At this point, it is hard to explain the difference in findings 
for reading and math« It may be that the more linear structure of 
math makes it easier to implement the cbmpohehts of ihstructibhal 
skills and make it easier to observe than is reading* Reading tends 
to be taught in several small groups and the four model components may 
hot be implemented in each group or it may be harder for the observer 
to follow and record the many changes of activities in a reading 
class • in the latter case, the error is iti the measurement* 

Fortunately i this analysis using the same instruments will be 
conducted for data collected in 1983, 1984 and 1984-1985. Gohtrol 
classrooms will be included iti these studiesi If the findings of this 
study are replicated ^ we will have more confidence in the results t and 
conclude that Hunter's Instructional Skills model is effective for 
math arid riot for reading; At this time, it is premature to make such 
a conclusion and our finding must be regarded as teritative. 
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